Disaster Survival Guide in Petascale Computing: An Algorithmic Approach

1 Disaster Survival Guide in Petascale Computing: An Algorithmic Approach 3 Jack J. Dongarra, Zizhong Chen, George Bosilca, and Julien Langou 1.1 FT-MPI: A fault tolerant MPI implementation . . . . . . . . 6 1.1.1 FT-MPI Overview . . . . . . . . . . . . . . . . . . . . 6 1.1.2 FT-MPI: A Fault Tolerant MPI Implementation . . . 6 1.1.3 FT-MPI Usage . . . . . . . . . . . . . . . . . . . . . . 7 1.2 Application Level Diskless Checkpointing . . . . . . . . . . . 8 1.2.1 Neighbor-Based Checkpointing . . . . . . . . . . . . . 10 1.2.2 Checksum-Based Checkpointing . . . . . . . . . . . . . 11 1.2.3 Weighted-Checksum-Based Checkpointing . . . . . . . 13 1.3 A Fault Survivable Iterative Equation Solver . . . . . . . . . 17 1.3.1 Preconditioned Conjugate Gradient Algorithm . . . . 17 1.3.2 Incorporating Fault Tolerance into PCG . . . . . . . . 18 1.4 Experimental Evaluation . . . . . . . . . . . . . . . . . . . . 21 1.4.1 Performance of PCG with Different MPI Implementations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 1.4.2 Performance Overhead of Taking Checkpoint . . . . . 22 1.4.3 Performance Overhead of Performing Recovery . . . . 24 1.4.4 Numerical Impact of Round-Off Errors in Recovery . . 26 1.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 1.6 Conclusion and Future Work . . . . . . . . . . . . . . . . . . 28